Digital Image Processing Laboratory : The EM Algorithm

This laboratory explores the use of the expectation-maximization (EM) algorithm for the estimate of parameters. In particular, we will use the EM algorithm for two applications: clustering, and hidden Markov model training. You are encouraged to implement your solutions to this laboratory in Matlab. For an original derivation of the monotone convergence of the likelihood for the EM algorithm, the reader is encouraged to read [1]. The publication [2] has a clear treatment of the application to the EM algorithm to Gaussian mixtures and to novel approach to the problem of order identification. The paper [3] is a widely cited tutorial reference, and the papers [3] and [4] present valuable analyses of the convergence properties of the EM algorithm. The EM algorithm is very useful for computing maximum likelihood (ML) estimates of parameters when observations are incomplete. Consider the case when it is necessary to estimate the parameter vector θ ∈ Ω from the complete data Y and X. The ML estimate of θ is given by